{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Short-term load-forecasting and Cross-Validation Demo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from metric_funcs import mape\n",
    "from sklearn.metrics import mean_squared_error as mse\n",
    "from forecaster import Forecaster as FC\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### A crucial step in building this model is to define an appropiate cross validation methodology to fine tune it.\n",
    "\n",
    "Within conventional libraries I couldn't find a cv-methodology that fitted this idea of training multiple independent models at a given time-resolution so I built one (sklearn library does have one although the methodology is different; it aggregates the training data from but the k-sample to k+1 sample,  as opposed to partitioning equally spaced models).\n",
    "\n",
    "In this case I strive for independent models to avoid training long term trends and seasonalities; __my hypothesis is  that short-term forecasting is more correlated to closer past intervals__ (ie. hours before forecast, 1 day before, 1 week before). In any case, I believe we could test this hypothesis through cross-validation and improve the outcome.\n",
    "\n",
    "How the implement cross-validation works:\n",
    "\n",
    "1) Partitions the training set into n tasks given a user-defined resolution to resample the training data (ex. '1W', '3W', '1Y') -- __Its like defining k-folds but instead of k-splits, you provide a time split__.\n",
    "\n",
    "2) Under the hood it resamples the dataset, retrieves indices and generates n-tasks with equal size. \n",
    "\n",
    "3) Then trains-tests on each sample to get a score (in this case I chose MSE from sklearn library).\n",
    "\n",
    "Note: The last day in each task is used to test that sample as well as to train the next task."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1) Load dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "forecaster = FC('../data/train_cleaned.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2) Use cross-val method which takes:\n",
    "\n",
    "a) an error function to evaluate the testing, b) resampling resolution (spliting) and, c) parameters to initialize the RF Regressor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mape scores:  [8.733619782424094, 6.557545639293015, 6.123784267677421, 8.10295064974378, 17.388288443982752]\n"
     ]
    }
   ],
   "source": [
    "print 'Mape scores: ', forecaster.cross_val(mape, '3W', n_estimators = 10, max_features=4)[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This method returns a list of forecasting errors where each represents a 1-day forecast at a 15-min interval horizon.\n",
    "\n",
    "As the result shows, the model is performing relatively poor but we can consider this as a benchmark model. \n",
    "The __next natural steps would be to tune and find the best set of parameters, features, training times__, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3) To illustrate this process, lets plot MSE vs. a varying parameter (ie. number of trees, max random parameters per split or training time per task); for this example I will vary the training time on each task."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Tuning for training time\n",
    "\n",
    "def tuning(res = '3M'):\n",
    "    '''\n",
    "    Tune parameters\n",
    "    \n",
    "    Params\n",
    "    ------\n",
    "    res: str\n",
    "        Resolution to resample dataset.\n",
    "    features: int\n",
    "        num of max_features parameter to cross-validate.\n",
    "        \n",
    "    Return\n",
    "    ------\n",
    "    Plot MSE vs. variable\n",
    "    '''\n",
    "\n",
    "    error = [] \n",
    "\n",
    "    for res in resolutions:\n",
    "        # Compute the mean MSE.\n",
    "        RF_e = np.array(forecaster.cross_val(mse, res, max_features = 3))\n",
    "        error.append(np.median(RF_e))\n",
    "        \n",
    "    return error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x11f9b7110>"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfkAAAEZCAYAAACU8lxmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XeYFfXZ//H3BxEQKxZsKCKgokbF2E101ViiiVhiFKMx\nT3iMT+zRX6IYI9gSTayJJTEmRrGXKLbYs7E3BAVBxYIiKqKCgkSk3L8/vrNyWHdhd9mzc87s53Vd\ne+2cOTNz7tmFvefbFRGYmZlZ8XTIOwAzMzMrDyd5MzOzgnKSNzMzKygneTMzs4JykjczMysoJ3kz\nM7OCcpI3a4CkHSWNzjuOxkg6VdJlrX1sniR1kDRdUo8WnNtb0rxyxNWWJA2S9O+847DikMfJW7WS\nNB2o+we8NDALmJvtOyIibsgrtoWR9Bjw14i4Ju9YikJSb+C1iFgih89eApgNrBMR7yzmtQYBP4qI\nnVslOGv3OuYdgFlLRcSydduS3gQGRUTVl4IkLRERc/OOoy1Vyz03EqeY/7BpVlFcXW9Foexr/g5p\nmKTTSl7vIumtktcTJf1C0kuSpkq6TtKSzT02e3+wpPez4/5X0jxJa38tSOkcYFvgz5I+k3SBpCWy\n438uaTwwLjv2T9n1pkl6RtK2Jdc5U9Lfs+3e2fmHZsdPlnRSC49dStK12T2OkXRS6c+h3r3UxX20\npDclfSjpd/WO+V9J4yR9LOmeuqr4hu65ZN/a2THLZ7F8mF3/5JLrdpB0oaSPJL0O7NFQjPV+f7+S\nNDaL5a/1fn97SxqV3fejkjaqd+7/k/QSMKOBy/8n+z42+53uK2nF7H4/zD7vTkmrl1xzkKS3suNf\nl/TDRuK+UNK/JS2zsPsza4yTvLU39UtcBwC7AOsCWwCHNvdYSd8DjgJ2BNYDdm7g3HTBiJOBp0jN\nCctFxAklb38/u+43stdPAxsDKwK3AreUJqYGbJvFtgdwelaF3dxjzwRWA9YGdgcOaexeSuwNbAZ8\nE/iBpB8DSNofODG7r1WAZ4Dr651b/55LP+tyoAuwDunnPkhS3e/nSOA72XlbAg0myXoOzq7Tl/Rz\nHZzFuSXwF+CnpJ/134HhkkprOg8k/TxWaOC6O5AeMPtlv9PbSX9brwB6AD2BL4GLs89bFjgf2CUi\nlgO2B14qvWD2EPP3LNbdI6KhhwuzRXKSt/buwoiYEhFTgbtJyaq5xx4A/C0iXouI/wKnN+Fz1cC+\nsyPi04iYBRAR12Wv5wHnAcsBfRq5XgBDImJ2RIwEXgY2bcGxBwBnRcT0iJgEXNKEe/ldRHwWEROB\nPwIDs/1HAL+NiNeze/gtsFVpibb+PZP9XLIEewBwUkTMjIi3gAuZ/xB2AOn38X72+zinCXFenB3/\nSRZLXZyHA5dFxAuR/CPbv2XJuRdl586icV/9TiPio4gYHhFfZgn6HNJDYJ15wDckdY6IyRHxSsl7\nnYGbgK7AgIj4sgn3ZtYgJ3lr7yaXbM8EFlYt2tixawATS96bSMNJfFHeLX2RVS+PkzQV+IT0R3/l\nxk6OiCmNxNecY1evF0fpfTUl7rdJPw9IJdhLJX0i6RNgCjCHVLpt6NxS3Ul/n0o7sr0NrJlt1/+Z\nv72YcZ5UF2f2816t5LMWFmeDJC0t6UpJb0uaBjxM9ruLiOmkB4yjgQ+yqvy+JaevD+wJnF4N/RSs\nsjnJW5F9TkqMdVZv7MDF9D4LJq61WXgVd2PvfbVfUg3wC2DfiOgWEd1I99OSh4fm+ICv38uirFXv\n+Pey7YmkzpArZl/dImKZiHiu5PjGfhYfkkZK9CzZ1xOYlG2/X+9zS49rSpw968V5egNx3tqEOBt7\n75fZZ2wRESuQmnDmnxBxf0TsSnqYeAP4c8nbLwE/A+5fRJOL2SI5yVuRjQL2krRCVkV8TJk+52ZS\ne/F6kroCpy7i+Mmk9vCFWZY0LOsTSZ0knc6CDyz1NSf5L+zYm4FTsk5vPUht34vyq+z4tYFjgRuz\n/X8GTpW0AUD2e9i/KQFGxBxSP4TfZqXiXsDxwLCSOI+XtIaklYBfNeGyR5ccf3JJnH8FjpK0RRbn\nMpK+J2mpJsY6D/iIBX+ny5JqSD7NPm9I3RuSViu5/hzSw9sCY/wj4jpgKPCQpHWaEodZQ5zkrSga\nKk39A3iFVDV7L1B/3Hxzhj01emxE3E3qJPYo8CrwePZWY+23FwEHZ1XD5zVy/XtJVbzjgTeBaaTS\na1Pja07Js/T1EFIpegJwH6lteGHt0AB3kR6oRgC3RcTVAFlJ+HxSh8Fp2TG7LSLG0n1HkR50JgD/\nBq6KiLokfznp5zOa1KHvlkXECOn3/xDpZzoO+F0W5zPAz4HLs2aFV4AfLSLO+oYAN2S/031I970C\n8DHp38M9JccuQSrpv0dqwtg2u9cFRMTfgXOBh9WCCYLMoMyT4UjqTPrD14k0Jv/WiDhd0hBSZ5cP\ns0NPiYj7snMGk3q5zgGOi4gHyhagWRlI2hgYERGd845lcUk6mtT5a9cG3mu1SWDKTdJE0iQzj+Yd\ni1lbKutkOBExS9JOETEz+4PwhKR/ZW9fEBEXlB4vqR9pKEw/UrvgQ5L6hqflswqXld7uIVXTngPc\nkW9ELSNpDVJb8tPABqR+Aect9CQzq1hlr66PiJnZZmfSQ0Vdwm6oXXAAcGNEzImICaRqta3KHaNZ\nKziK1C77Gqkt9uh8w2mxzqQ26s+A+0nV4Fcs5PhqeQCvljjNWlXZp7WV1IHUVtcbuDQinpO0J6kT\nzKHA88CJEfEpacjKUyWnT2LBYSxmFamh6uxqlI1H37iJx84ltS9XvIhoyigBs8Jpi5L8vIjoT6p+\n30rShsBlwLoRsRlpyM755Y7DzMysvWmzBWoi4jNJtcAe9dri/0rqnQup5F46lrUH88fFfkWSq97M\nzKzdiYhmzZVR1pK8pJUlLZ9tLwXsCrwiabWSw/YDxmTbdwIHZeOCe5Gm8Hy2oWtHRNV/DRkyJPcY\nfB/FuYei3EcR7sH3UVlfRbiHiJaVbctdkl8duDprl+8A3BQR90q6RtJmpAkgJpDmuCYixkq6GRhL\nGppzZLT0zszMzNq5cg+hGw1s3sD+Hy/knN+RTVJhZmZmLecZ73JUU1OTdwitogj3UYR7gGLcRxHu\nAXwflaQI99BSZZ3xrlwkuRbfzMzaFUlEJXW8MzMzs/w4yZuZmRWUk7yZmVlBOcmbmZkVlJO8mZlZ\nQTnJm5mZFZSTvJmZWUE5yZuZmRWUk7yZmVlBOcmbmZkVlJO8mZlZQTnJm5mZFZSTvJmZWUE5yZuZ\nmRWUk7yZmVlBOcmbmZkVlJO8mZlZQTnJm5mZFZSTvJmZWUE5yZuZmRWUk7yZmVlBOcmbmZkVlJO8\nmZlZQTnJm5mZFZSTvJmZWUE5yZuZmRWUk7yZmVlBOcmbmZkVlJO8mZlZQTnJm5mZFZSTvJmZWUE5\nyZuZmRWUk7yZmVlBOcmbmZkVVFmTvKTOkp6RNFLSaElDsv3dJD0g6VVJ90tavuScwZLGSxonabdy\nxmdmZlZkiojyfoDUNSJmSloCeAI4Ftgf+Dgifi/pJKBbRJwsaUPgOmBLoAfwENA36gUpqf4uMzOz\nQpNERKg555S9uj4iZmabnYGOQAADgKuz/VcD+2TbewM3RsSciJgAjAe2aui6s2eXK2IzM7NiKHuS\nl9RB0kjgA+DBiHgOWDUiJgNExAdA9+zwNYGJJadPyvZ9zdVXN7TXzMzM6nQs9wdExDygv6TlgNsl\nbUQqzS9wWHOve+KJQ5kwATp2hJqaGmpqahY/WDMzswpRW1tLbW3tYl2j7G3yC3yY9BtgJvC/QE1E\nTJa0GvDviOgn6WQgIuLc7Pj7gCER8Uy968Teewc77QTHH99m4ZuZmeWm4trkJa1c13Ne0lLArsA4\n4E7gJ9lhhwHDs+07gYMkdZLUC+gDPNvQtc88E845B6ZPL+MNmJmZVbFyt8mvDvxb0ijgGeD+iLgX\nOBfYVdKrwC7AOQARMRa4GRgL3Asc2Vg3+k02gV12gYsvLvMdmJmZVak2ra5vLXVD6F5/HbbZBl57\nDVZcMe+ozMzMyqfiquvLrU8f2H9/+P3v847EzMys8lR1SR7g3Xdh001hzBhYffWcAzMzMyuTlpTk\nqz7JA5x4IsyaBZdckmNQZmZmZdRuk/yUKbDBBvD889CrV46BmZmZlUm7a5Ovs8oqcPTRcPrpeUdi\nZmZWOQpRkgf49FPo2xdqa2HDDfOJy8zMrFzabUkeYPnl4Ze/hNNOyzsSMzOzylCYkjzAzJmpND98\nOGyxRQ6BmZmZlUm7LskDdO0Kp56avszMzNq7QiV5gEGD0gx4//lP3pGYmZnlq3BJvlOn1Mv+17+G\nKmyJMDMzazWFS/IABx8MU6fCv/6VdyRmZmb5KWSSX2IJOOusVJqfNy/vaMzMzPJRyCQPsM8+0LEj\n3Hpr3pGYmZnlo1BD6Op78ME0E97LL6eEb2ZmVq3a/RC6+r7zHVhjDbjmmrwjMTMza3uFLskDPPkk\nDByYhtV17lzmwMzMzMrEJfkGbLcdbLIJXHFF3pGYmZm1rcKX5AFefBH22ANefx2WXrqMgZmZmZWJ\nS/KN2HRT2HFH+OMf847EzMys7bSLkjykNvntt0/fu3UrU2BmZmZl4pL8Qqy3HgwYAOedl3ckZmZm\nbaPdlOQB3nkH+veHsWNh1VXLEJiZmVmZtKQk366SPMDxx6eFay6+uJWDMjMzKyMn+SaYPBk23BBe\neAF69mzlwMzMzMrEbfJNsOqq8POfwxln5B2JmZlZebW7kjzAtGnQty88/jisv34rBmZmZlYmLsk3\n0QorwIknwmmn5R2JmZlZ+bTLkjzA55+n0vw996Qe92ZmZpXMJflmWHppOOUUOPXUvCMxMzMrj3ab\n5AEOPzytNf/443lHYmZm1vradZLv3BmGDk0l+ipstTAzM1uodp3kAQ45BKZMgQceyDsSMzOz1tXu\nk3zHjnDmmS7Nm5lZ8bT7JA+w337p+z//mW8cZmZmramsSV5SD0mPSHpZ0mhJx2T7h0h6V9IL2dce\nJecMljRe0jhJu5UzvjodOsDZZ6ee9nPntsUnmpmZlV9Zx8lLWg1YLSJGSVoGGAEMAA4EpkfEBfWO\n7wdcD2wJ9AAeAvrWHxTfGuPk64uAHXeEQYPgsMNa9dJmZmaLreLGyUfEBxExKtueAYwD1szebijQ\nAcCNETEnIiYA44GtyhljHSmV5ocOhVmz2uITzczMyqvN2uQlrQNsBjyT7Tpa0ihJV0paPtu3JjCx\n5LRJzH8oKLtvfxv69YMrr2yrTzQzMyufNknyWVX9rcBxWYn+MmDdiNgM+AA4vy3iaIqzzkol+s8/\nzzsSMzOzxdOx3B8gqSMpwQ+LiOEAETGl5JC/Andl25OAtUre65Ht+5qhQ4d+tV1TU0NNTU2rxLv5\n5vCtb8Ell8BJJ7XKJc3MzJqttraW2traxbpG2ReokXQN8FFEnFCyb7WI+CDb/gWwZUQcLGlD4Dpg\na1I1/YO0Uce7Uq+8kqrux49PK9aZmZnlrSUd78rdu3574FFgNBDZ1ynAwaT2+XnABOCIiJicnTMY\nGATMJlXvf20uunIneYCf/hTWXDNNlGNmZpa3ikvy5dIWSf7tt1PV/bhx0L17WT/KzMxskZzkW9kx\nx8CSS8IFFyz6WDMzs3Jykm9lH3wAG20Eo0bBWmst+ngzM7NycZIvg8GD4eOP4Yor2uTjzMzMGuQk\nXwaffALrrQdPPQV9+7bJR5qZmX1NxU1rWwQrrgi/+AUMGZJ3JGZmZs3jknwTzJgBffrA/ffDppu2\n2ceamZl9xSX5MllmmdQ2/5vf5B2JmZlZ07kk30RffJHa5m+6Cbbdtk0/2szMzCX5curSJbXLn3JK\nWnvezMys0jnJN8Nhh8GkSfDww3lHYmZmtmgLTfKSDinZ3r7ee0eXK6hK1bFjmsvepXkzM6sGiyrJ\nn1Cy/ad67/20lWOpCgccAF9+CcOH5x2JmZnZwi0qyauR7YZetwsdOsDZZ8Opp8LcuXlHY2Zm1rhF\nJfloZLuh1+3GnnvC8svDDTfkHYmZmVnjFjqETtJM4HVSqb13tk32et2IWLrsETYcV5sPoavvP/9J\na86PGwedOuUaipmZtQOtPne9pJ4LOzki3m7Oh7WWSkjyALvvDvvuC//3f3lHYmZmRVf2BWokrQTs\nALwTESOaGV+rqZQk//zzMGAAvP46LLVU3tGYmVmRtfpkOJLulrRxtr06MIbUq36YpONbHGlBbLEF\nbLMNXHpp3pGYmZl93aKq61+OiI2y7VOADSLix5KWBZ6IiE3aKM76cVVESR5g7FioqUml+eWWyzsa\nMzMrqnJMazu7ZHsX4F6AiJgOzGteeMW04Ybw3e/CBRfkHYmZmdmCFlWSvwt4AHgX+DvQKyKmSVoK\neL6ulN/WKqkkD/DWW6nq/tVXYeWV847GzMyKqBwl+UHARsBPgAMjYlq2fxvgqmZHWFC9esFBB8E5\n5+QdiZmZ2XxearaVvP8+bLwxvPgi9OiRdzRmZlY05Rgnf+fCTo6IvZvzYa2lEpM8wEknwaefwp//\nnHckZmZWNOVI8lOAicANwDPUm68+Iv7TgjgXW6Um+Y8/hvXXh6efhj598o7GzMyKpBxJfglgV2Ag\nsAlwD3BDRLy8OIEurkpN8gBnnAGvvQbXXpt3JGZmViRlnfFOUmdSsv8DcHpEXNL8EFtHJSf56dNT\nKf7hh1MbvZmZWWsoS5LPkvtepAS/DnAn8PeImNTCOBdbJSd5SGPmH3sMbr8970jMzKwoylFdfw2w\nMWkSnBsjYszihdg6Kj3J//e/sN56cNttsNVWeUdjZmZFUI4kPw/4PHtZeqCAiIhcJnKt9CQPcMUV\ncMst8OCDeUdiZmZFUPZV6CpFNST52bOhX7+U7HfeOe9ozMys2pVjxjtroSWXTD3tf/1rqPDnETMz\nKygn+TI66CD4/HO4++68IzEzs/bISb6MOnSAs85Kpfl5XrPPzMzamJN8mX3/+9C1K9x0U96RmJlZ\ne+OOd23gkUfgiCNg7NjUVm9mZtZcFdfxTlIPSY9IelnSaEnHZvu7SXpA0quS7pe0fMk5gyWNlzRO\n0m7ljK+t7Lwz9OwJ//hH3pGYmVl7UtaSvKTVgNUiYpSkZYARwADgf4CPI+L3kk4CukXEyZI2BK4D\ntgR6AA8BfesX26utJA/wzDPwgx/A+PHQpUve0ZiZWbWpuJJ8RHwQEaOy7RnAOFLyHgBcnR12NbBP\ntr03aWa9ORExARgPFGLOuK23hm9+Ey6/PO9IzMysvWizjneS1gE2A54GVo2IyZAeBIDu2WFrkpa2\nrTMp21cIZ54J55yTFrExMzMrtzZJ8llV/a3AcVmJvn5de3XVvbfQN74Bu+4KF12UdyRmZtYedCz3\nB0jqSErwwyJieLZ7sqRVI2Jy1m7/YbZ/ErBWyek9sn1fM3To0K+2a2pqqKmpaeXIy2PoUNhmGzjy\nSFhppbyjMTOzSlVbW0ttbe1iXaPsQ+iylew+iogTSvadC3wSEec20vFua1I1/YMUpONdqf/7P1h+\neTj33LwjMTOzalFxC9RI2h54FBhNqpIP4BTgWeBmUqn9beCHETEtO2cwMAiYTaref6CB61Z1kp80\nCTbZBEaPhjXWyDsaMzOrBhWX5Mul2pM8wNlnw5/+BCeckKrul1km74jMzKySVdwQOmvcr38NDz0E\nI0ZA797udW9mZq3PST5HG2+c5rR/5BF48cWU7H/7W/jss7wjMzOzInCSrwAbbQQ33AD/+Q+8/DL0\n6ZNWr3OyNzOzxeEkX0H69YPrroPHHoNXX00l+zPOgE8/zTsyMzOrRk7yFWj99WHYMHjiCXjjjZTs\nhw6FadPyjszMzKqJk3wFW289uPpqePppePvtVI0/ZAhMnZp3ZGZmVg2c5KtAnz5w1VVpJbt334W+\nfeE3v4FPPsk7MjMzq2RO8lWkd2/429/guefggw9Ssv/1r+Hjj/OOzMzMKpGTfBXq1Qv++tc0xv6j\nj1K1/uDBadvMzKyOk3wVW2cd+Mtf4IUXUqe89deHk06CKVPyjszMzCqBk3wB9OwJl18Oo0bBjBkp\n2f/ylzB5ct6RmZlZnpzkC2StteDSS+Gll+CLL9K4+xNPTO33ZmbW/jjJF1CPHmnxm9GjYc4c2HBD\n+MUv4P33847MzMzakpN8ga25Jlx8cZoqF9L0uccdB++9l29cZmbWNpzk24HVV4cLL4SxY6Fjx7Qw\nzjHHpDH3ZmZWXE7y7chqq8H558O4cdClC2yyCRx1FEycmHdkZmZWDk7y7dCqq8If/gCvvALLLAOb\nbQY//zm8807ekZmZWWtykm/HuneHc89NK96tsAL07w9HHAETJuQdmZmZtQYneWPlleF3v4PXXkvb\n3/wmHH44vPVW3pGZmdnicJK3r6y0Epx9Nowfn9rvt9wSBg1Ky92amVn1cZK3r1lxRTjzzJTse/SA\nrbeGn/wEXn8978jMzKw5nOStUd26wemnp+Teqxdssw38+MepWt/MzCqfk7wt0gorwJAhqdq+b1/Y\nfns45JDUO9/MzCqXIiLvGJpNUlRj3EXx2Wdp2tyLL4Zddknt9jvtBEsskXdkZrY4Zs2CMWNg5Mi0\nuuWECXDYYXDAAdDBRcLcSSIi1KxzqjFZOslXhunT4cor4brrYNIk+OEPYeDA1IavZv0zNLO2NmMG\nvPhiSuZ1Sf2116B3b9h88zSktnt3+OMf07FDhsD++zvZ58lJ3nLz2mtwww3pa/ZsOOggOPjgNF++\nmeXr449TIq9L5iNHpsmvNt44JfO6pP6Nb8BSSy14bgT8618pyc+aBUOHwj77ONnnwUnecheR1rW/\n/nq48cbUeW/gwJT0e/XKOzqzYotIC1CVJvMXXoCpU9PMlnXJfPPNYYMNYMklm3fte+5JyX7u3JTs\nBwxwrV1bcpK3ijJvHjz+eCrd33pr6rQ3cGCq1l911byjM6tuEfDmmwsm85Ej0/+70mTev3+qgm+t\nkncE3HVXSvZSSvbf/76TfVtwkreKNXs2PPhgSvh33QVbbZWq8/fdF5ZfPu/ozCrbnDlpNEtpMh85\nMv3fqZ/Q11yzbRJuBNxxR0rynTql73vu6WRfTk7yVhVmzoS7704J/5FHUg/9gw+Gvfb6enugWXvz\nxReph3tpCX3MmDQxVWky798/TUOdt3nz4PbbU5Lv2jV932MPJ/tycJK3qjNtGvzzn6kNf8QI2Hvv\nVKW/yy7Nay80q0bTp6c+LKUl9PHjU9NWaQl9001h2WXzjnbh5s2D225LE2gts0z6vttuTvatyUne\nqtr778PNN6cS/ptvprG5AwfCdtu5J69Vv48++nqHuEmTUo/20hL6xhtDly55R9ty8+bBLbekJN+t\nWyrZf+c7TvatwUneCuONN1Lv/OuvT6WdgQPT16ab+o+FVb6pU+GxxxZM6p999vUe7uuvDx075h1t\necydCzfdBGecAauskpL+Tjv5/+/icJK3womA0aPnj8FfaqnUfj9wIPTpk3d0ZguaOxeuuCL1PN98\n8/lf/funIaTtsUZq7tz0f/eMM2D11VOyr6nJO6rq5CRvhRYBTz2V/mDcfDP07JmS/YEHwhpr5B2d\ntXfPPgtHHpk6n116aaqGt/nmzEk1c2ecAWutlZL9DjvkHVV1cZK3dmPOnNQz/4Yb0jCe/v1Twt9/\n/7RUrllb+fhjGDw4DQ39/e/T4k2ukm7cnDkwbFhaznrddVOy3377vKOqDi1J8mWtPJL0N0mTJb1U\nsm+IpHclvZB97VHy3mBJ4yWNk7RbOWOz6taxY+q5e9VVqcPe0UfD/fenKtG9907J//PP847Simze\nvLR2w4Ybpo5y48bBoYc6wS9Kx47wP/8Dr76aHswPOST9X37qqbwjK6ayluQlfQuYAVwTEZtk+4YA\n0yPignrH9gOuB7YEegAPAX0bKrK7JG+N+eyzVLK//np4+uk0OcfAgbD77mnCDrPWMGIEHHVUamO/\n7LLUoc5a5ssv4eqr4eyz01S7p5+eFrmyr6u4knxEPA5MbeCthoIcANwYEXMiYgIwHtiqjOFZAS23\nHPz4x3DffWnRnO23h3PPTW32P/sZ1NamjkBmLTF1akrue+0FRxyRpm12gl88nTrB4Yen/6/77puG\nzu61Fzz3XN6RFUNefT2PljRK0pWS6iY1XROYWHLMpGyfWYt0757+ID/+eCp59e4Nxx8Pa68NJ5wA\nzz+fOvOZLcq8efCPf6Sq+QgYOzZVObfH3vLl0qlTenAaPz4l+f32S3PijxiRd2TVLY8RmpcBZ0RE\nSDoLOB/43+ZeZOjQoV9t19TUUOMxGbYQPXvCSSelr7FjU5v9QQel9tOBA9OwvA02yDtKq0Qvvpge\nFmfPTp3rttgi74iKrXPnNErhpz9NfR723jv9zIcOTR1s25Pa2lpqa2sX6xpl710vqSdwV12bfGPv\nSToZiIg4N3vvPmBIRDzTwHluk7fFFpGqBG+4IU2806NH6gR00EFeJc/g00/htNPSv42zzoJBg1xy\nz8MXX6S5B845J7XVDx2aJsVqjyquTT4jStrgJa1W8t5+wJhs+07gIEmdJPUC+gDPtkF81k5JaTW8\nCy+EiRNTx58RI9IsZHvumTrvuYd++xMB114L/frBf/8LL7+c2oyd4PPRpQsce2yaBXOHHdLiNz/4\nQZokyxat3L3rrwdqgJWAycAQYCdgM2AeMAE4IiImZ8cPBgYBs4HjIuKBRq7rkryVzeefpx76116b\nhvUMGJBK+DvvDEsskXd0Vk5jxqSq+RkzUq959/KuPDNnwuWXwx/+kJL+kCGw0UZ5R9U2PBmOWSv7\n4INUXXvttfDee6n9/tBDPYd+0UyfnqqBhw1L3484wg90le7zz9OD2HnnpQfw005LtS9FVqnV9WZV\na7XVUo/855+Hhx9Oc+fvs0+asvScc+Cdd/KO0BZHRHqI69cPPvkkleSPPNIJvhosvTT88pepGn/T\nTWHHHeFHP0qT7Nh8LsmbNdO8efDEE6l0f+utsMkmqTp///1hhRXyjs6aaty4NFPiRx+lEqGnVq1u\n06fDn/7DcI85AAAN5UlEQVQEF12UJr867TTo2zfvqFqXS/JmbaBDB/j2t+Evf0nrgR9zDNxzTxqm\n98Mfwp13plm8rDLNmJGGUu6wQ+pvMWKEE3wRLLssnHIKvP566jy73XZw2GHpdXvmJG+2GLp0SZN2\n/POf8NZbsMsuqUPQmmumat+nnvKEO5UiAm67LU1o8957qXf2sccWdz339mq55eDUU1NyX3dd2Gab\n9PB9441pWGR74+p6szJ46600BG/YsDSJyiGHpK+iVR9Wi9deSzUukyalqnkvcdp+TJuWmtXuuAMe\nfTSV8AcMSJPsrFllc6q6d71ZhYlI1cHXXpsm3enVKyX7Aw+EVVbJO7rimzkTfvtb+POfU1XuMcfA\nkkvmHZXlZfr0tFrl8OGpia1v39SRdsCA1Pmy0kfMOMmbVbA5c+DBB1PCv+ee1K5/yCGpRLHUUnlH\nVywRqW/EccfBttvC+eenRYrM6syenUr2d9yRkn6XLvMT/jbbVOYICyd5syoxfTrcfntK+M89l/64\nHHpoGgZUiX9cqskbb6S29jffhEsuSf0kzBYmAkaOTAn/jjtg8uT08L3PPunfT5cueUeYOMmbVaH3\n3kudgoYNgylT0mI5hxyShuZZ0/33v2lZ4UsugV/9Ks1v0KlT3lFZNXrzzVS6v+MOGDUKdt01Jfy9\n9oJu3fKLy0nerMqNGQPXXZe+unVLyX7gwLR4jjXunntS6X3zzeGCC2CttfKOyIrio4/g7rtTwn/k\nEdhyy/nV+muv3baxOMmbFcS8eam98Npr0/C8/v1Tdf5++6UhQpZMmJBK7GPHpolQdt8974isyGbO\nTP1q7rgjLTu89top4dfNglnujntO8mYF9MUXqSQxbBjU1sJ3v5sS/m67td+e4rNmpfkILroITjgB\nTjwxrUNu1lbmzEkzX9ZV60fMT/jbb1+e+Rec5M0K7qOP4JZbUsJ//fU0FO+QQ9KSuZU+/Ke13H9/\nGgq30UZpmeB11sk7ImvvItLkSnUJ/+234XvfSwl/t92ga9fW+RwnebN25I03Utv9tdem1z/6Uaoy\n7N49jcFfZZU0l35R1kF/551Uah85Ev74x9QJyqwSvfNOSvjDh8Ozz8JOO6WE/73vLd78GE7yZu1Q\nRBqGd+ONqVfwhx+mXvpTpqTlOFdeeX7SL30AaOh1t26V91Dw5ZepM91556US/K9+5XkFrHpMnZo6\nhg4fDg88AJttljrt7bNPmna3OZzkzWwBs2alKv66pD9lyoIPAaXbU6akxVtWWqlpDwXdu5f/oeDh\nh+Goo6B371R67927fJ9lVm5ffJH+TQ8fniZr6t59fsLffPNFN7k5yZvZYvnyywUfCuo/BNR/QJgx\nA1ZcsWkPBKusko5tykPBpEmpM90zz8DFF8P3v99++hxY+zB3bvr3XTcBzxdfpIQ/YECaFKuhTrVO\n8mbWpmbPnv9QsLAHgrrXn32WEv3CHgjefDP1nP/5z2Hw4NbrtGRWqSLglVfmJ/zx42HPPVMJf/fd\n0zK64CRvZhVu9mz4+OOFPxB07gxDh8J66+UdrVk+3nsvVeffcQc8+WRa52KffeBnP3OSNzMzK4xP\nP4X77ksJ/8YbneTNzMwKqSXV9RU2WMbMzMxai5O8mZlZQTnJm5mZFZSTvJmZWUE5yZuZmRWUk7yZ\nmVlBOcmbmZkVlJO8mZlZQTnJm5mZFZSTvJmZWUE5yZuZmRWUk7yZmVlBOcmbmZkVlJO8mZlZQZU1\nyUv6m6TJkl4q2ddN0gOSXpV0v6TlS94bLGm8pHGSditnbGZmZkVX7pL8VcDu9fadDDwUEesDjwCD\nASRtCPwQ6Ad8F7hMUrPWza02tbW1eYfQKopwH0W4ByjGfRThHsD3UUmKcA8tVdYkHxGPA1Pr7R4A\nXJ1tXw3sk23vDdwYEXMiYgIwHtiqnPHlrSj/8IpwH0W4ByjGfRThHsD3UUmKcA8tlUebfPeImAwQ\nER8A3bP9awITS46blO0zMzOzFqiEjneRdwBmZmZFpIjy5lhJPYG7ImKT7PU4oCYiJktaDfh3RPST\ndDIQEXFudtx9wJCIeKaBa/rBwMzM2p2IaFZftY7lCqSEsq86dwI/Ac4FDgOGl+y/TtKFpGr6PsCz\nDV2wuTdpZmbWHpU1yUu6HqgBVpL0DjAEOAe4RdJPgbdJPeqJiLGSbgbGArOBI6Pc1QxmZmYFVvbq\nejMzM8tHJXS8a7KGJtepNpJ6SHpE0suSRks6Nu+YWkJSZ0nPSBqZ3ceQvGNqKUkdJL0g6c68Y2kp\nSRMkvZj9Phps5qoGkpaXdEs2IdbLkrbOO6bmkrRe9nt4Ifv+aTX+P5f0C0ljJL0k6TpJnfKOqSUk\nHZf9jaqav7fNnUhuYaoqydPw5DrVZg5wQkRsBGwLHCVpg5xjaraImAXsFBH9gc2A70qq1nkNjiM1\nE1WzeaQOrf0jolp/DwAXA/dGRD9gU2BczvE0W0S8lv0eNge+CXwO3J5zWM0iaQ3gGGDzrNN0R+Cg\nfKNqPkkbAYOALUh/p74nad18o2qSJk8ktyhVleQbmVynqkTEBxExKtueQfojVpXzAUTEzGyzM+mP\nQNW1/UjqAewJXJl3LItJVNn/5/okLQd8OyKuAsgmxvos57AW13eANyJi4iKPrDxLAEtL6gh0Bd7L\nOZ6W6Ac8ExGzImIu8CiwX84xLVIzJ5JbqKr+o1DtJK1Derr82jDBapBVc48EPgAejIjn8o6pBS4E\nfkkVPqDUE8CDkp6TdHjewbRQL+AjSVdlVd1XSFoq76AW04HADXkH0VwR8R5wPvAOaWKyaRHxUL5R\ntcgY4NtZVXdX0gP9WjnH1FKNTSS3UE7yOZG0DHArcFxWoq86ETEvq67vAWydrT9QNSTtBUzOalbq\nD/WsNttn1cN7kpqAvpV3QC3QEdgcuDS7l5mkKsqqJGlJ0nTdt+QdS3NJWoFUcuwJrAEsI+ngfKNq\nvoh4hTRc+0HgXmAkMDfXoFpPkwomTvI5yKq/bgWGRcTwRR1f6bIq1X8De+QdSzNtD+wt6U1SaWsn\nSdfkHFOLRMT72fcppPbfamyXfxeYGBHPZ69vJSX9avVdYET2O6k23wHejIhPsmrufwLb5RxTi0TE\nVRGxRUTUANOA13IOqaUmS1oVIJtI7sOmnFSNSb7aS1wAfwfGRsTFeQfSUpJWruvdmVWp7gq8km9U\nzRMRp0TE2hGxLqlT0SMR8eO842ouSV2zmiEkLQ3sRqqmrCpZVeRESetlu3ahujtEDqQKq+oz7wDb\nSOqSrQa6C1XYCRJA0irZ97WBfYHr842oyRqbSA4WnEhuodpixrtW09DkOnWddKqFpO2BHwGjs/bs\nAE6JiPvyjazZVgeultSB9LB4U0Tcm3NM7dWqwO3ZdM8dgesi4oGcY2qpY0kzXy4JvAn8T87xtEjW\n/vsd4Gd5x9ISEfGspFtJ1duzs+9X5BtVi90maUXmT7JW8Z05mzOR3CKv5clwzMzMiqkaq+vNzMys\nCZzkzczMCspJ3szMrKCc5M3MzArKSd7MzKygnOTNzMwKyknerAJJWrFkudL3Jb1b8rpJ81tky1X2\nXcQxR0oa2Arx7ifplpLXv5E0ruT1PpJua+G1H5O0yeLGaNYeVdVkOGbtRUR8AvQHkHQaMCMiLqh/\nnCRFI5NdRMSgJnzOZYsba+YJ0jKxdbYBZkjqFhFTSVOiPtFKn2VmTeSSvFnl+2pqS0m9Jb0s6VpJ\nY4DVJP1F0rOSRks6teTYxyRtImkJSVMl/U7SKElPSFo5O+ZMSceWHP87Sc9IGidpm2x/V0m3Shoj\n6ZZspbsFStbZlLSzsqlDIa2QdSewbfb6qyQvaQ9JT0p6XtINdSvNSdpCUm12/XvqpiMtuZ8OkoZJ\nOi27p2skvSjpJUlHt9pP26xAnOTNqs/6wPkRsXG2MM1JEbEVadni3SRt0MA5ywP/jojNgKeBnzZ2\n8YjYGvgVaSpNgGOA9yNiY+DM7HMa8iSwXbYa4bjsc7bLpqjdCBiRJe6TgZ0jYgtgNHCcpE6kmoD9\nImJL4DrgrJJrdyLNA/9SRJwBfBNYOSI2jYhNgKpcWMis3Fxdb1Z93oiIkSWvf5TNZ92RtKbAhnx9\nsaCZJfPZjwAaW4r2nyXH9My2v0WaN5uIeEnSy42c+yRpZb9lgKeAZ0kJ/ZvA6IiYI2m7LL4ns4VP\nlgQeB/qRHgQeyvZ3ACaWXPtK0pz8f8hevw6sJ+ki4N4qnqvfrKyc5M2qz+d1G5L6kBZ12SIipksa\nBnRp4JwvS7bn0vj//VlNOKaxVSCfAA4HugJ/iohPJS0L7EB6AKg7918RcdgCF5Q2A16MiB0Xcu1d\nJF0cEV9GxCdZk8F3gSMl7R8RRzRyrlm75ep6s+pTmmSXAz4jdXJbHdi9Cec01xPAgQCSvkEqdTdk\nDKn0vzXwUrbvJdJKbHWd7p4EdpTUK7te1+xBZSywpqQts/1LZtX+df4CPAzcmLXHrwx0iIjbSM0K\n/Rfj/swKy0nerPp81Zs+Il4gtX+PA/5Bqvr+2nH1thd53Xr+BKyRdfT7DSkhf/q1k1Mv/+eAyREx\nL9v9FNCLrCQfER8Cg4CbJI0iJf++EfEl8APgAkkvAi8AW5XGlVXVvwxcBawFPJot1/x3YHAT7s+s\n3fFSs2a2UJKWADpGxKys1H0/KTHPW8SpZpYzt8mb2aIsAzxcMgnPz5zgzaqDS/JmZmYF5TZ5MzOz\ngnKSNzMzKygneTMzs4JykjczMysoJ3kzM7OCcpI3MzMrqP8PMKSLBlW+lYwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11c80ab10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot MSE vs. training time.\n",
    "fig = plt.figure(figsize=(8,4))\n",
    "ax = fig.add_subplot(1,1,1)\n",
    "resolutions = ['1W','2W','3W','4W','5W','6W', '7W', '8W', '9W', '10W']\n",
    "x = np.linspace(1,len(resolutions), len(resolutions))\n",
    "ax.plot(x, tuning(resolutions), label = 'MSE')\n",
    "ax.set_ylabel('MSE')\n",
    "ax.set_xlabel('Training Weeks')\n",
    "ax.set_title('Tuning training period per task')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The model error seems to stabilize at around 5-6 weeks of training, this is a good starting point..."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
